The invention relates to reducing noise in a sequence of video frames and more particularly relates to such techniques by using impulse reducing techniques.
One application of the invention is to remove noise from a sequence of video frames which have been digitized from the analog domain. The benefit of removing noise is that the efficiency of a digital compression system is increased, thus resulting in better perceptual quality. Digital compression systems generally make use of redundant information in a sequence of video frames in order to reduce the amount of data necessary to restore the frames of the video sequence. The removal of this redundant information and subsequent encoding of the sequence of frames produces a compressed bit stream representing the original video sequence. The quality of the restoration of the decompressed bit stream into the original sequence of video frames depends on the efficiency with which the compression system encodes the information, and on the ratio between the amount of original video data and the compressed bit stream data. So, if a sequence of video frames is given, the higher the compression ratio is the smaller is the produced bit stream. As the compression ratio increases a point is reached in which non-redundant information is lost or degraded in the compression process, so that perceivable and therefore objectionable artifacts are produced.
In digital images fine details are represented as high frequency two-dimensional information, whereas coarse details are represented by low frequency two-dimensional information that may even include DC frequency, i.e. zero frequency.
Image or video compression systems require more bits to encode fine image details than coarse details, so fine details produce larger bit streams.
Some fine image details are caused by non-redundant information, but there are also fine details caused by random noise in the original input sequence of frames, which can be introduced in the analog domain as well as in the digital domain. For example in the analog domain noise can be caused by recording and playback of the sequence from a video tape, by errors introduced in transmission or by interference created by external signal sources. In the digital domain, random noise can be created in the analog-to-digital conversion process or by thermal noise in electronic components, electronic interference and the like.
The invention described in this specification relates to two types of noise, namely temporal noise and salt-and-pepper noise. Both types can be described as random. Temporal noise occurs, if a pixel value in a current frame differs substantially from its value in the previous frame, while the values of the adjacent pixels didn't change that much between the two frames. The second type of noise is also known as impulsive noise or speckle noise. It occurs if the value of one pixel in a current frame differs substantially from the values of the adjacent pixels in the same frame. In contrast to temporal noise this second type of noise is defined with respect to a single frame only, that is the definition has no time component.
The compression system cannot decide if high-frequency information is noise and therefore irrelevant for the image content or if it is important for the frame. Thus the compression system processes all high-frequency content. In case that this is caused by noise then it causes degradation of the frame quality because bits are wasted for encoding the noise, which could have been used to contain actual information. So in order to increase the efficiency of the compression system it is desirable to reduce the amount of random noise in the original sequence of frames before compression, so that all bits of the compressed bit stream represent actual information. Furthermore, the noise itself is a visible artifact so it is preferred to remove it.
A simple way used in the prior art to reduce high-frequency content of video sequences is the use of a low-pass filter on an input video sequence. The low-pass filter reduces and even eliminates high frequencies depending on the cut-off property of the filter, which simply cuts off all frequencies that exceed a threshold frequency. However, this filter also cuts off actual high-frequency information and hence produces a ‘soft’ image.
Another well known way to reduce the high-frequency noise is to use a two-dimensional spatial filter, e.g. a median filter, which preserves some high-frequency image details like borders or edges of objects. However the detection of borders or edges can be affected by noise.
Furthermore there are filters known that take the motion of objects into account. These filters either use motion estimation information or motion detection, both information can be derived from the luminance or chrominance information in the frames of a video sequence.